Interaction and Focus: Towards a Coherent Degree of Detail in Graphics, Captions and Text

Direct manipulation is the key concept of interaction with systems that use graphics to communicate with the user. The latest generation of such systems supports also interaction with other modalities, i.e., text and figure captions. However, the effects of interaction within one modality must be reflected in the other modalities in order to keep the presentation consistent. Although some approaches exist that distribute the content of a presentation coherently on different modalities, none of them is suited for highly interactive systems. This paper provides a fast and simple solution to solve this problem. We argue that interaction with objects changes their actual importance for the presentation. Therefore a central focus structure is used as a common basis for the generation of graphics and figure captions. A semantic network is used to determine relevant background objects and concepts. We will show how graphical abstraction can be used to render images that have a desired focus structure and to generate figure captions appropriately.